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PREFACE 


An  experiment  was  conducted  in  which  subjects  had  to  land  a  simulated  F-16 
aircraft  using  a  CAVE  (Cave  Automatic  Virtual  Environment)  facility,  a  three 
dimensional  virtual  environment  room  consisting  of  multiple  mirrors,  3-D  video- 
projected  displays  in  a  highly  stressful  environment.  This  performance  study  was 
accomplished  when  the  six  subjects  had  a  haptic  stick  (force  reflecting  joystick)  condition 
being  either  on  or  off.  The  tracking  mission  studied  herein  also  had  the  constraint  that 
the  participants  were  required  to  land  the  aircraft  in  only  one  attempt.  During  the  data 
collection,  the  visual  scene  was  partially  obscured  by  clouds  in  the  early  part  of  the 
mission,  thus  reducing  the  available  visual  field.  Hence,  it  was  hypothesized  that  the 
subjects  may  benefit  more  from  sensory  information  provided  by  the  haptic 
manipulandum  when  the  visual  field  was  reduced.  The  main  results  of  this  study 
indicate  that  performance  can  be  enhanced  in  the  CAVE  by  the  haptic  condition.  One 
negative  aspect  of  this  study  was  that  the  CAVE  scenario  was  found  to  be  so  extremely 
compelling  that  in  two  situations  subjects  could  not  complete  the  experiment  due  to 
simulator-induced  sickness  instigated  by  their  exposure  to  the  highly  immersive  visual 
field  which  became  quite  overwhelming. 
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INTRODUCTION 


Synthetic  environments  provide  a  valuable  forum  to  immerse  the  user  into  a 
virtual  reality  computer  application  to  such  a  degree  as  to  offer  perceptual  power,  which 
may  compensate  for  the  fact  that  an  operator  has  to  perform  a  task  at  a  remote  location. 
Traditional  user  interfaces  are  two-dimensional  and  some  loss  of  immersive  fidelity  may 
arise  due  to  distractions  and  other  confounding  variables  that  compete  for  sensory 
modality  attention.  A  CAVE,  however,  is  an  interactive,  real  time,  three-dimensional 
visual/audio/haptic  environment,  which  extrapolates  and  expands  the  immersive 
properties  of  such  synthetic  environments  to  a  novel  degree.  With  the  appropriate  high- 
resolution  visual  display,  auditory,  and  haptic  rendering,  the  goal  is  to  provide  to  the  user 
a  higher  sense  of  presence  about  a  remote  environment.  A  subject  immersed  in  the 
virtual  world  of  a  CAVE  can  be  overwhelmed  by  its  commanding  size  and  its  3-D 
rendered  visual  scene  which  utilizes  stereoscopic  surrounding  view  (with  perspective 
correction)  working  with  other  senses,  in  harmony,  to  emulate  a  synthetic  world  from  the 
standpoint  of  the  virtual  visitor’s  eyes,  ears,  and  hands.  To  resolve  and  create  the 
compelling  3-D  view,  the  subject  wears  stereoscopic  glasses  to  turn  the  projections 
(overlap  of  two  images)  into  hologram-like  images. 

Literature  Review 

Over  thirty-five  years  ago  the  concept  of  an  Ultimate  Display  was  discussed 
(Sutherland,  1965)  in  which  each  of  the  senses  would  be  stimulated  in  order  to  provide  a 
complete  immersion  of  the  operator  into  a  virtual  world.  Some  synthetic  environments  of 
this  type,  however,  may  not  be  practical  unless  they  impart  some  benefit  to  performance 
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for  specific  tasks.  The  development  of  a  CAVE  is  a  generalization  of  traditional 
two-dimensional  display  systems  and  can  be  attributed  to  the  early  work  at  the  University 
of  Illinois  at  Chicago  (Cruz-Neira  et  al.,  1992,  1993).  The  first  systems  consisted  of  four, 
large  rear-projection  displays  to  form  walls  on  a  10-foot  cube.  A  front  and  two  sidewall 
projections  are  rendered  with  a  floor  projection  realized  by  a  ceiling  system  (Figures  1-2). 
All  views  are  reflected  off  single  surface,  silvered,  mirrors  excited  by  a  rear  projector.  It 
is  necessary  to  provide  3-D  visual  images  and  appropriate  depth  perception.  These  views 
are  constructed  by  the  overlap  of  two  images  and  then  resolved  to  the  observer  via  a 
stereoscopic  display  and  the  use  of  lightweight  3-D  LCD  shutter  glasses  (cf.  Figures  3a- 
b).  It  is  also  necessary  to  provide  the  proper  perspective  realization  for  distance 
correction  and  to  maintain  a  location  and  orientation  vector  of  the  forehead  of  the  user. 

An  alternative  means  of  providing  improved  immersion  over  2-Dimensional 
presentations  includes  studies  in  head  mounted  displays  (HMD),  which  has  been  a  useful 
means  for  enhancing  fidelity  (Durlach  and  Mavor,  1995).  In  these  systems,  the  left-  and  , 
right-eye  images  are  generated  on  separate  displays  and  directly  placed  in  front  of  the 
eyes,  or  somehow  fed  through  optics  to  appear  in  front  of  the  eyes  (Figure  4).  Some  of 
the  difficulties  with  HMDs,  however,  include  having  to  deal  with  the  latency  of  both  the 
image  updating  and  monitoring  of  head  position.  Such  a  system  also  produces  strain  on 
the  neck  muscles  with  continued  operation.  The  CAVE  overcomes  these  two  drawbacks 
to  the  HMD  (i.e.  latency  and  fatigue)  at  the  expense  of  physical  space  and  portability. 

The  main  attribute  of  the  CAVE,  over  traditional  interactive  synthetic  environments,  is 
that  it  provides  even  greater  immersion  properties.  Improving  presence  (defined  as  a 
strong  impression  or  feeling  that  you  are  at  the  remote  environment)  can  be  achieved  via 
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Fig.  1  -  Diagram  of  CAVE  Facility 


Side  Projection 
(from  mirrors) 


Fig.  2a  -  Top  View  of  Visual  Rendering  System  for  CAVE 


Fig.  2b  -  Side  View  of  Visual  Rendering  System  for  CAVE 
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Figure  3a  -  Subject  in  CAVE  Environment 


Figure  3b  -  Subject  in  Tracking  Scenario 


Right  eye  image  of  virtual  object 


Left  eye  image  of  virtual  object 


Left  Eye 


Figure  4.  Stereoscopic  Viewing  to  Produce  3-D  Image  to  Subject 


gleaning  information  from  alternative  sensory  modalities.  Part  of  the  definition  of  virtual 
reality  has  as  its  goal  the  creation  of  a  believable  representation  of  some  actual 
phenomenon.  Presently,  CAVE  facilities  exist  at  a  number  of  research  organizations  and 
universities  performing  exploratory  investigations.  Some  of  the  studies  include  the 
designing  of  the  interiors  for  automobiles,  inspection  of  dangerous  areas  such  as  inside 
boilers,  for  studying  molecular  docking  (i.e.  finding  how  molecules  bind  together),  for 
educational  purposes,  and  also  for  certain  psychological  studies  (e.g.  mitigation  of 


phobias,  etc.). 


Pertinent  Related  Research  for  the  Performance  Task 


A  major  cause  of  aircraft  accidents  can  be  attributed  to  tight  landing  conditions 
and  turbulence  manifested  by  wind  shear,  wind  burst,  or  other  difficult  environmental 
conditions  (Phillips,  1995  a,  b,  and  Mecham  &  Kong,  1995).  Since  an  increasing  number 
of  simulation  systems  being  developed  today  may  involve  unmanned  air  vehicles  (e.g. 
aircraft  flown  at  a  remote  location  with  no  pilot  onboard  under  control  of  an  operator  in  a 
safe  environment,  (Cupp,  1998)),  there  necessitates  the  need  to  study  how  remote  crew 
stations  may  be  enhanced  to  improve  the  performance  of  the  operator  in  certain 
undertakings  as  well  as  increasing  the  operator’s  presence  about  the  external  environment 
at  a  significant  distance  from  the  mission.  Studies  involving  haptic  (including  force 
reflecting  joystick  interfaces)  devices  have  shown  that  motor  learning  can  be  expedited 
via  kinematic  feedback  (Schmidt  and  Young,  1991).  Both  performance  and  learning 
show  improvement  depending  on  how  subjects  locate  objects  in  a  spatial  sense,  (Delmez- 
De  Jager  &  Schepens,  1995  and  Bryant  &  Lanca,  1995).  The  response  strategies  elicited 
by  operators  are  known  to  be  predicated  on  sensory-sets  or  motor-sets,  (DeFranceso  «fe 
DeSario,  1995). 

A  prior  study  of  investigating  tracking  performance  with  aircraft  during  difficult 
landing  conditions  (Repperger,  et  al.,  1997)  demonstrated  that  by  adding  certain  force 
characteristics  to  a  joystick,  several  performance  measures  were  significantly  improved 
when  the  force  condition  was  active.  Such  a  study  was  previously  conducted  in  a  CAVE- 
like  simulator  termed  the  FTTE,  which  was  a  precursor  to  the  synthetic  environment 
discussed  here.  Subjective  comments  by  the  pilots  in  the  prior  1997  study  indicated  that 
they  would  tend  to  “feel  the  direction  of  the  runway”  (via  the  haptic  interface)  when 
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limited  visual  conditions  were  available.  Thus  the  level  of  situational  awareness  or 
presence  was  improved  via  the  haptic  interface  concerning  the  remote-tracking  scenario, 
when  visual  conditions  were  less  than  ideal.  Haptic  interfaces  are  well  known  to  improve 
tracking  performance  when  a  subject  has  reduced  neuromotor  capabilities  (Repperger,  et 
al.,  1995a, b). 

Objectives 

The  intention  of  this  study  was  to  investigate  if  similar  performance  improvements, 
which  occurred  in  a  less  compelling  virtual  environment  (Repperger,  et  al.  1997),  may 
also  appear  in  this  new  synthetic  CAVE  surroundings  where  the  degree  of  immersion  is 
even  more  pronounced.  To  summarize  the  results  from  the  1997  study,  it  was  shown  that 
the  benefits  derived  using  the  haptic  interface  tended  to  ameliorate  performance  through 
the  mitigation  of  lateral  landing  error  (reduced  error  in  landing  near  a  desired  touch-down 
point),  producing  less  subjective  workload,  and  lessening  the  amount  of  stick  activity  in 
handling  the  aircraft.  It  was  also  noted  that  the  haptic  stick  seemed  to  show  its  efficacy 
best  when  the  turbulence  condition  was  active.  The  active,  haptic,  stick  seemed  to  give 
the  operator  a  sense  of  presence  about  the  orientation  of  the  landing  path.  This  study  will 
consider  the  influence  on  landing  performance  of  an  active  haptic  stick  used  in  a  CAVE 
environment  as  stated  by  the  following  hypothesis. 
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Hypothesis 

The  null  hypothesis  we  wish  to  reject  is  that  the  haptic  stick  provides  no  performance 
advantage  (using  a  variety  of  dependent  measures)  in  such  a  difficult  task  environment. 
Thus  it  is  desired  to  reject: 

Ho:  Haptic  sticks  do  not  influence  landing  performance  in  a  CAVE  environment. 

METHODS 

Subjects 

A  total  of  eight  subjects  participated  in  this  experiment  and  data  were  obtained 
from  the  six  subjects  who  were  able  to  complete  all  the  experimental  conditions 
discussed  herein.  A  subject  panel  from  a  local  contractor  for  the  Wright-Patterson  Air 
Force  Base  in  Ohio,  USA  provided  five  of  the  participants.  These  adult  people  were 
either  housewives  or  students  at  a  local  university  being  employed  part  time.  The 
compensation  for  participation  in  this  experiment  was  about  $6  dollars  (US)  an  hour  for 
their  participation.  The  sixth  subject  was  a  US  Air  Force  civil  servant. 

Apparatus  -  The  CAVE  Facility 

The  CAVE  (CAVE  Automatic  Virtual  Environment)  is  a  multi -person  10  foot 
room-sized,  high-resolution  3D  video  and  audio  environment,  which  can  be  used  to 
provide  compelling,  immersive  virtual  environment  experiences  (Figure  1).  This  unique 
facility  is  the  property  of  Wright  State  University  (Department  of  Psychology)  but  is 
located  at  Wright  Patterson  Air  Force  Base  in  Dayton,  Ohio,  USA.  In  addition  to  the 
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scope  of  the  academic  investigations,  the  US  Air  Force  also  has  the  opportunity  of 
performing  specific  experiments  related  to  military  applications.  Some  of  the  attributes  of 
the  CAVE  facility  include: 

(1)  The  visual  environment  consists  of  up  to  300°  surround,  plus  floors  (4  walls)  for 
projection  of  the  landing  task  to  the  subjects.  The  walls  are  10  x  10  x  10  foot  screens 
with  three  rear-projected  walls  and  a  top-projected  screen  for  the  floor  (Pyramid 
Video,  Figures  2a-b).  Imagery  is  created  by  an  SGI  (Silicon  Graphics,  Inc.)  Onyx 
computers  with  Infinite  Reality  graphics  boards  to  generate  similes  that  are  projected 
onto  the  walls  and  floor.  The  images  are  displayed  by  CRT  projectors  (M8500, 
Electrohome).  With  the  addition  of  the  unique  fourth  wall,  this  system  provides 
complete  immersion  in  the  azimuth  dimension  (the  only  non-imaged  area  being 
directly  overhead)  allowing  the  operator  to  monitor  and  interact  with  the  environment 
to  the  rear  as  well  as  to  the  sides  and  the  front.  To  give  the  illusion  of  3D,  two 
alternating  images  are  displayed  on  each  wall  at  a  rate  of  96  Hz  (48  Hz  refresh  rate 
per  field.  Figures  3a-b). 

(2)  Virtual  spatial  zed  audio  (provided  to  the  subjects  via  earphones)  is  also  available  at 
this  facility  but  was  not  used  in  this  investigation. 

(3)  Virtual  or  synthetic  objects  and  real  objects  are  integrated  via  careful  monitoring  of  the 
head  position  of  the  operator  (using  a  magnetic  tracking  system  employed  to  follow  the 
position  and  orientation  of  the  user’s  head  and  hand  (six  degree  of  freedom  Flock  of  Birds, 
Ascension  Technology)).  Stereo  images  are  interpreted  by  use  of  LCD  shutter  glasses 
(Crystal  Eyes,  Sterographics)  that  enable  a  different  image  to  be  displayed  to  each  eye  by 
synchronizing  the  rate  of  alternating  shutter  openings  to  the  screen  update  rates  (Figure  4). 
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(4)  The  CAVE  has  a  panoramic  view  that  varies  from  180°  to  300°  depending  on  the 
distance  of  the  viewer  from  the  projector  screens.  One  final  attribute  in  this  facility 
includes  the  user’s  finger-pinch  gestures,  which  are  sensed  to  provide  a  means  to  interact 
with  the  virtual  environment  (PinchGloves,  Fakespace,  cf.  Figure  3a,  not  used  in  this 
investigation). 


Experimental  Procedures 

The  subjects  trained  by  practicing  an  aircraft  landing  task  (four  sets  of  trials  as  in 
Figure  5a)  with  full  visual  information  (no  wind  turbulence)  until  50%  or  more  successes 
(good  landings)  were  achieved.  A  data  day’s  run  consisted  of  24  practice  trials  (Figure 
5a),  which  were  then  followed  by  24  trials  for  data  collection  with  the  cloud  cover 
condition  (Figure  5b).  A  subject  was  rerun  until  he/she  had  achieved  at  least  50% 
success  for  landings  starting  in  the  cloud  cover.  The  Task  Position  levels  (P  =1,2,  and  3) 
in  Figures  5a-b  refer  to  the  initial  position  of  the  F-16  and  will  be  described  in  the  sequel. 


The  Performance  Task 

The  duration  of  the  performance  task  was  fixed  to  not  exceed  120  seconds  in  length 
with  the  objective  of  landing  an  F-16  aircraft  successfully  on  a  runway.  The  subject  had 
but  one  opportunity  to  perform  this  task  at  an  optimum  landing  point  designated  on  the 
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p=l  P=2  P=3 

P  =  Initial  Position 


Figure  5  a.  Training  Conditions 

(No  Clouds  or  Turbulence) 


Figure  5b.  Data  Collection  -  1  Day  Runs 
(Full  Cloud  Condition) 
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ranway  (Figures  6,7).  There  were  three  levels  of  initial  position  based  on  the  original 
starting  point  being  aligned  with  the  runway  or  displaced  near  or  far  from  the  center,  to 
the  right  or  left.  Obstacles  were  located  to  the  right  and  left  of  the  landing  path,  as 
indicated.  Sometimes  the  subjects  landed  near  or  off  the  right  runway  but  such  a  landing 
was  deemed  successful  if  the  weight  on  wheels  at  touchdown  was  considered  safe.  As 
indicated  in  Figure  5b,  in  half  the  cases  (during  data  collection),  wind  turbulence  was 
present.  During  the  data  collection,  all  trials  started  in  the  clouds  in  which  the  runway 
could  not  be  initially  seen.  Subjects  were  told  that  they  must  have  the  nose  of  the  aircraft 
pitched  upward  slightly  at  landing  (3°  to  5°).  The  weight  on  wheels  at  touchdown  must 
be  distributed  on  the  two  back  wheels.  If  a  crash  occurred,  the  screen  would  immediately 
indicate  this  event  (or  a  successful  landing)  and  display  the  negative  downward  velocity 
at  the  touchdown  with  the  forward  speed.  In  Figure  7,  the  optimum  touchdown  point 
would  correspond  to  a  zero  longitudinal  error  (y=0),  zero  lateral  error  (x=0),  and  for  the 
elevation  above  the  runway  (z=6)  feet  due  to  the  height  of  the  landing  gear. 

Haptic  and  Thrustmaster  Sticks 

A  force-feedback  manual  control  stick  (IE-2000,  Immersion  Corporation)  was 
integrated  into  the  system  for  manipulating  objects  and  controlling  the  virtual 
environment  by  the  right  hand  of  the  subject  (Figure  3b).  Figure  8  illustrates  the  force 
reflection  algorithm  employed.  Here  the  force  reflected  back  on  the  operator  was  devised 
to  be  a  parabolic  function  of  the  heading  error  between  the  aircraft’s  body  axis  and  the 
runway.  Such  a  methodology  was  employed  in  the  1997  study  (Repperger,  et  al.,  1997) 


13 
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Figure  6.  Landing  Scenario  on  the  Right  Runway 
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Figure  7.  Performance  Metrics  Considered  at  Landing 
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Force  Reflected  in  Pounds  Against  the  Operator’s  Hand 


0.91  pounds 


Orientation  Angle  Between  Flight  Path  of  Aircraft  and  Runway 
Figure  8.  Haptic  Stick  Force  Reflection  Algorithm 
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and  had  demonstrated  efficacy  in  improving  the  presence  of  the  operator  on  the  final 
orientation  of  the  runway  when  the  visual  field  was  partially  occluded. 

A  Thrustmaster  joystick  was  utilized  by  the  subjects  with  their  left  hand  (Figure  3b) 
to  manipulate  aircraft  speed.  This  manipulandum  operated  as  a  displacement  device  with 
small  physical  effort  required  to  change  its  linear  position. 


Experimental  Design 

The  experimental  design  was  a  two- variable,  full  factorial,  and  repeated  measures. 
This  implies  each  subject  was  exposed  to  all  treatment  conditions  (within  subject  design, 
hence  each  subjects  acts  as  his  own  control).  The  first  independent  variable  (two  levels) 
was  the  haptic  stick  being  either  “on”  or  “off”  (“off’  corresponds  to  a  displacement  stick 
with  no  force  reflected.  The  “on”  condition  for  the  haptic  stick  was  described  previously 
and  represents  a  haptic  algorithm  to  provide  force  feedback  to  the  operator,  cf  Figure  8). 
The  second  independent  variable  (two  levels)  was  difficulty  of  the  task,  which  for  this 
analysis  consisted  of  low  and  high  levels  of  initial  position  error  of  the  aircraft  in 
relationship  to  the  orientation  of  the  runway.  As  discussed  in  the  results  section,  the 
initial  position  conditions  P=1  and  P=2  were  combined  into  a  low  difficulty  task.  The 
high  difficulty  task  consisted  of  the  conditions  P=3  for  the  initial  starting  points  (right  or 
left)  of  the  aircraft.  Another  variable  considered  during  the  data  collection  was  a  wind 
turbulence  factor  being  either  high  or  low.  For  this  task  difficulty  versus  haptics  study, 
all  data  utilized  both  levels  of  the  wind  turbulence  condition  but  with  easy  and  hard  initial 
starting  positions.  A  variety  of  dependent  performance  measures  were  considered.  At 
landing,  the  operator  had  to  reach  an  optimum  point  “sweet  spot”  on  the  runway  as 
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indicated  by  a  green  dome  on  the  visual  simulation.  The  root  mean  square  error  in  the  x 
and  y  direction  from  the  optimum  landing  point  (denoted  as  xrms  and  yRMs)  was 
calculated  when  the  F-16  aircraft  touched  down.  If  these  measures  were  too  large,  then 
the  landing  was  not  considered  successful,  since  the  aircraft  was  off  the  runway.  Crashes 
were  recorded  at  landing  if  too  much  weight  on  wheels  was  determined  based  on  the 
vertical  descent  velocity.  A  metric  for  calculating  crashes  was  developed  from  known 
flight  test  standards  (e.g.  an  F-16  would  be  considered  crashed  if  its  horizontal  velocity, 
at  landing,  exceeded  9  ft/sec).  Crashes  could  also  occur  if  the  aircraft  was  out  of  bounds 
during  touchdown  or  simply  ran  out  of  time  and  had  not  landed.  A  number  of  other 
dependent  landing  performance  measures  were  analyzed  including  the  aircraft’s  angular 
rates  at  landing  for  both  the  pitch  and  roll  axis  of  the  aircraft  as  well  as  higher  derivatives 
of  these  position  and  orientation  variables  at  touchdown. 

For  both  training  and  data  collection,  two  levels  of  task  difficulty  were  selected 
based  on  varying  the  initial  point  of  starting  the  task  (i.e.  away  from  the  correct  alignment 
of  the  runway)  with  the  constraint  of  having  a  fixed  amount  of  time  to  land  the  aircraft. 
Figure  5a  illustrates  an  experimental  design  scenario  during  the  training  phase.  In  Figure 
5b,  the  data  collection  scenario  is  portrayed.  The  P  =  1,  2,  or  3  initial  starting  points  were 
presented  randomly  to  the  right  or  left  when  the  simulation  was  started.  The  amount  of 
initial  lateral  displacement  error  was  either  zero,  +1000  feet,  or  +2000  feet.  Preliminary 
analysis  had  shown  little  difference  in  task  difficulty  as  a  consequence  of  being  initially 
either  to  the  right  or  left  of  the  runway  for  the  same  amount  of  displacement. 
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Training 

Table  I  describes  the  chronological  pattern  of  runs  for  a  typical  day’s  run.  Six 
runs  were  conducted  with  the  stick  being  either  active  (haptic  feedback)  or  passive.  The 
next  six  runs  reversed  the  condition  of  the  stick.  Typically  subjects  ran  this  pattern  for 
about  fifty  minutes  and  the  initial  positions  were  randomized.  The  next  training  day  the 
subject  ran  the  initial  haptic  stick  condition  being  reversed  and  they  continued  to  practice 
a  pattern  similar  to  Table  I. 

A  critical  performance  measure  was  the  event  of  a  crash  at  landing.  A  crash  was 
defined  as  having  too  much  weight  on  wheels  as  the  aircraft  landed  based  on  known 
military  standards.  A  crash  would  occur  if  any  of  the  following  events  happened:  (1)  The 
weight  on  wheels  exceeded  the  maximum  allowed,  (2)  If  one  back  wheel  (rather  than 
both  back  wheels)  touched  down  and  the  weight  on  that  side  of  the  aircraft  exceeded  the 
safe  amounts  for  landing,  or  (3)  If  the  front  wheels  should  touch  down  first  (with 
sufficient  negative  downward  velocity/acceleration).  Thus  for  a  successful  landing,  the 
pitch  attitude  (orientation)  of  the  aircraft  must  be  slightly  nose  up  (i.e.  pitch  angle  up 
about  3°  to  5°)  at  landing  and  not  having  an  excessively  large  downward 
velocity/acceleration  of  the  aircraft  before  the  actual  touchdown. 

If  subjects  were  successful  with  the  training  paradigm  displayed  in  Table  I,  they 
then  moved  on  to  the  instrumented  flying  (i.e.  flying  in  clouds)  for  data  collection.  Table 
n  illustrates  a  typical  day’s  run  for  data  collection  for  the  same  haptic  condition  for  12 
runs  consisting  of  two  replications  of  the  three  initial  positions  and  two  turbulence 
conditions  presented  randomly.  To  reach  this  level  of  skill  it  typically  took  two 
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additional  weeks  of  practice.  Data  were  not  collected  and  subjects  continued  to  practice 
until  at  least  50%  successful  landings  for  the  Table  II  trials  were  achieved. 


In  the  data  collection  phase  in  Table  H,  the  cloud  condition  was  initially  presented 
and  the  subjects  had  to  fly  instruments  until  they  broke  out  of  the  clouds  (at  1.8  nautical 
miles).  After  breaking  out  of  the  clouds,  they  had  only  one  chance  to  land  (with  about  36 
seconds  remaining).  When  the  haptic  stick  condition  was  active  the  subjects  could  “feel”  the 
direction  of  the  runway  prior  to  breaking  out  of  the  clouds.  If  the  haptic  stick  condition  was 
not  active,  the  subjects  had  no  clue  as  to  the  direction  of  the  runway  and  could  only  sense  the 
lateral  displacement  error  when  they  immediately  broke  out  of  the  clouds  with  their  vision. 
They  still  had  only  1.8  nautical  miles  to  land  the  aircraft  (in  about  36  seconds). 

Table  I  -  Training  Table  -  No  Clouds  -  Full  Visual  Field  (Pos.=Initial  Position) 


Pos. 

Pos. 

Pos. 

Pos. 

Pos. 

Pos. 

P=3 

P=2 

P=1 

P=3 

P=2 

P=1 

P=1 

P=3 

P=2 

P=2 

P=3 

P=1 

P=2 

P=1 

P=2 

P=3 

P=1 

P=3 

||||[|[|§[QQ|QQ|| 

P=3 

P=2 

P=3 

P=1 

P=1 

P=2 

The  initial  positions  P  =  1,  2,  3  were  presented  randomly  (with  the  assistance  of  a  random 
number  generator)  so  that  each  subject  did  not  have  knowledge  as  to  the  initial  position  he 
may  have  had  in  starting  the  task.  In  each  set  of  six  runs  (same  haptic  stick  condition,  cf. 
Figure  5a),  three  levels  of  initial  positions  occurred  twice.  The  12  runs  during  the  data 
collection  phase  were  conducted  with  the  alternative  starting  stick  condition  on  successive 
days.  This  procedure  was  instigated  to  mitigate  any  ordering  or  fatigue  effects. 


Table  11  -  A  Possible  set  of  Runs  in  The  Clouds  (Reduced  Visibility)  Conditions 


(Initial  Position:  P=l,  1 

P=2,  P=3,  Turbulence  = 

1  or  0  (on  or  off)) 

Haptic 

Stick 

P/T 

P/T 

P/T 

P/T 

P/T 

P/T 

Haptic  Off 

P=l, 

T=0 

|SH| 

mSMM 

loHi 

mSMM 

Esi^g 

EB^H 

Haptic  Off 

JEBMi 

EoHI 

ISH 

Es^H 

E5IH 

Haptic  On 

EgHI 

iolH 

igng 

ESHi 

l^gi 

l^i 

IBIH 

eshh 

E5BB 

Haptic  On 

iSHl 

I3BI 

iS9Hi 

eshi 

EB^H 

fmjmm 

Some  possible  confounding  effects  may  be  due  to  varying  reinforcement  schedules  and 
inconsistent  delivery  of  critical  instructions  for  subjects  that  completed  the  landing  tasks. 
Both  the  reinforcement  schedules  and  the  rules  of  engagement  had  to  be  unswerving 
across  subjects  to  provide  reproducibility.  If  a  subject  performed  well,  he/she  was 
complemented  on  his/her  performance  but  not  given  additional  instruction.  If,  however, 
the  subject  continued  to  crash  at  landing,  they  were  told  to  focus  on  the  following  key 
points  at  touchdown  to  accomplish  a  successful  landing: 


Instruction  Set  Given  to  Subjects 

(1)  Both  back  wheels  should  touch  first;  hence  the  body  axis  of  the  aircraft  should  be 
pitched  up  slightly. 

(2)  The  air  speed  should  be  approximately  180  nautical  miles  per  hour  at  the  landing. 

(3)  A  gradual  angle  of  approach  (small  pitch  changes)  was  generally  correlated  with  a 
successful  landing,  especially  near  the  ground. 

(4)  The  negative  downward  velocity  should  not  be  in  excess  of  -5  feet/second  at  the 
touchdown. 
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(5)  The  roll  angle  of  the  aircraft  should  be  near  zero  (no  wing  could  touch  the  ground). 

(6)  The  speed  of  the  aircraft  (left  hand  control)  was  not  a  major  factor  in  a  successful 
landing  as  long  as  it  was  kept  near  180  nautical  miles/hour. 

If  the  subject  still  continued  to  crash  at  landings,  the  above  six  rules  were  reinforced 
and  practiced  until  steady-state  behavior  occurred  and  the  percent  of  successful  landings 
increased. 

Subjective  Assessments 

At  the  conclusion  of  a  CAVE  run,  it  was  of  interest  to  see  if  subjects  were 
possibly  influenced,  in  a  simulator-sickness  sense,  by  their  exposure  to  this  highly 
immersive  visual  field  rendering.  An  exit  questionnaire  was  given  to  the  subjects  and  it 
is  reproduced  in  Appendix  A  of  this  report.  Two  subjects  (out  of  the  eight  that  started), 
could  not  finish  the  experiment  due  to  simulator-sickness  effects. 

Data  Analysis 

At  the  landing  of  the  F-16  aircraft,  which  had  to  be  accomplished  in  one  attempt, 
the  data  recorded  included  the  key  experimental  variable  conditions  (haptic  condition  on 
or  off,  the  initial  position  level,  and  the  turbulence  on  or  off)  as  well  as  the  important 
dependent  measures:  X,  Y,  and  Z  landing  error  with  respect  to  the  optimum  point,  the 
velocity  and  acceleration  of  these  variables,  the  typical  aircraft  orientation  variables  (roll, 
pitch,  and  yaw),  their  respective  velocities  and  accelerations,  the  altitude,  position, 
velocity  and  acceleration  at  touchdown  and  whether  a  crash  occurred.  The  criteria  for  a 
crash  would  be  that  a  sufficiently  high  weight  of  wheels  was  recorded  at  touchdown,  the 
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aircraft  could  be  out  of  bounds,  or  had  an  untoward  attitude  variable  recorded  at  the 
instant  of  contact  with  the  ground.  The  data  were  analyzed  in  SYSTAT®  8.0. 

RESULTS 

The  risk  level  that  we  incorrectly  reject  the  null  hypothesis  (when  Ho  is  true) 
was  selected  as  a  =  .05.  This  gives  a  95%  confidence  in  our  decision  process.  Since  this 
was  a  within-subjects  design,  blocking  across  subjects  normally  occurred  which  reduced 
this  source  of  variation  (within  a  block,  however,  all  other  experimental  conditions  were 
randomized).  We  look  at  the  main  effects  and  study  their  respective  interactions. 

Six  different  performance  measures  are  reported  here  (cf.  Figure  7).  Since  an 
optimum  landing  point  was  displayed,  and  the  actual  touchdown  point  was  recorded,  this 
gave  rise  to  the  following  error  variables  at  touchdown:  (1)  lateral  landing  error  (X  from 
the  optimum  point),  (2)  longitudinal  landing  error  (Y  from  the  optimum  point),  (3)  crash 
or  no  crash  condition,  (4)  aircraft  Roll  error  (0°  is  ideal),  (5)  pitch  error  (excessive 
deviations  from  the  ideal  3°  to  5“  suggested),  and  (6)  vertical  downward 
velocity/acceleration  if  this  magnitude  digressed  significantly  from  the  recommended  -5 
feet/sec  velocity  at  touchdown  (-9  ft/sec  would  constitute  a  crash). 

For  all  the  analyses,  the  difficulty  of  the  task  ,was  first  dichomidized  into  two 
groups.  The  less  difficult  task  occurred  with  the  initial  position  at  0°or  at  a  position 
displaced  +  1000  feet  from  center.  The  high  level  of  difficulty  considered  the  +  2000  feet 
initial  displacement.  T-tests  were  then  conducted  on  the  landing  variables  across  both 
levels  of  task  difficulty  with  significant  differences  appearing.  Thus  the  two  levels  of 
difficulty  were  accepted  as  viable  independent  variables  to  perform  subsequent  analysis. 
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The  first  (more  general)  ANOVA  then  considered  haptics  versus  task  difficulty  for  the 
dependent  measures  consisting  of  some  key  landing  variables.  Table  in  illustrates 
significant  effects  for  the  normalized  dependent  measures;  crashes  Xrms,  Yrms,  and 
Zveiocity  at  the  touchdown  from  this  ANOVA.  In  Table  III,  the  p  value  and  F  value  are 
given  for  the  dependent  performance  measure  of  interest.  The  mean  values  of  the 
variables  depicted  in  Table  III  are  displayed  in  Figure  9. 


Table  III  -  ANOVA  Results  for  Various  CAVE-Haptic  Tasks  (p  levels) 
(*  indicates  significant  at  the  a  =.05  accepted  level  of  risk  -  Type  I  error) 


Treatment 

Crashes 

X-RMS 

Y-RMS 

Z-  Velocity 

Condition 

(percent) 

At  landing 

At  landing 

(RMS  Value) 

Difficulty 

p  =  0.230 

F=  1.603 

p  <  .001* 

F=  62.536 

p  =  0.006* 

F  =  10.077 

p  =  0.289 

F=  1.231 

Haptics 

p  =  0.009* 

F  =  9.552 

p  =  0.003* 

F=  12.949 

p  =  0.070 

F  =  3.793 

p  =  0.074 

F  =  3.834 

Difficulty  * 
Haptics 

(Interaction) 

p  =  0.788 

F  =  .075 

p  =  0.10 

F  =  0.880 

p  =  0.235 

F=  1.533 

p  =  0.445 

F  =  0.623 

To  explain  the  normalization,  the  percent  crashes  were  determined  as  a  ratio 
involving  the  crashes  with  each  block  (cf.  Table  II).  The  X-RMS  values  were  normalized 
to  one  runway  width  (46  feet).  The  Y-RMS  values  were  divided  by  150  and  expressed  in 
feet.  The  Z  values  were  divided  by  9  (negative  descent  velocity  more  negative  than  -9 
ft/sec  would  indicate  a  crash).  Figure  9  illustrates  the  means  of  these  normalized  variables 
for  the  corresponding  data  displayed  in  Table  III.  Since  both  haptics  and  task  difficulty 
showed  main  effects  but  their  interactions  were  not  significant  (p>.05),  further  analysis  of 
the  constituent  dependent  variables  was  then  conducted.  Using  a  subset  of  the  data  from 
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Table  III,  a  second  analysis  was  accomplished  in  which  it  was  desired  to  look  within  the 
difficulty  and  turbulence  conditions  using  some  of  the  above  dependent  variables  including 
the  rotational  variables  R  (roll  axis)  as  well  as  the  vertical  descent  variable  Z  (which  was 
related  to  the  occurrence  of  a  crash).  Again,  a  different  subset  of  the  data  from  Table  III 
was  then  applied  in  a  third  analysis  in  which  it  was  desired  to  look  within  the  haptics  and 
turbulence  conditions  using  some  of  the  above  dependent  variables  as  well  as  the  rotational 
variables  R  (roll  axis)  including  the  vertical  descent  variable  Z  (which  was  related  to  the 
occurrence  of  a  crash). 


Dependent  Performance  Measures 


Figure  9.  Normalized  F-16  Landing  Performance  Variables 
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Table  IV  -  ANOVA  Results  for  Various  CAVE-Haptic  Tasks  (p  levels) 
(*  indicates  significant  at  the  a  =.05  accepted  level  of  risk  -  Type  I  error) 


Treatment 

X-RMS 

Y-RMS 

|R-velocity|| 

|R-acce]eration| 

Condition 

At  Landing 

At  landing 

At  landing 

At  landing 

Difficulty 

p  <  0.01* 

F  =  57.099 

p  =  .014* 

F  =  8.368 

p  =  0.734 

F  =  0.121 

p  =  0.969 

F  =  0.002 

Turbulence 

p  <  0.01* 

F=  163.34 

P<0.01* 

F  =  32.665 

p  =  0.012* 

F  =  8.737 

p  =  0.016* 

F  =  7.911 

Difficulty  * 
Turbulence 
(Interaction) 

p  =0.198 

F=  1.861 

p  =  0.779 

F  =  0.083 

p  =  0.991 

F  =  0.001 

p  =  0.578 

F=  0.326 

Table  V  -  ANOVA  Results  for  Various  CAVE-Haptic  Tasks  (p  levels) 

(*  indicates  significant  at  the  a  =.05  accepted  level  of  risk  -  Type  I  error) 


Treatment 

Percent  Crashes 

X-RMS 

Y-RMS 

|R-velocity|| 

Condition 

At  Landing 

At  landing 

At  landing 

At  landing 

Haptics 

p  =  0.078 

F  =  3.721 

p  =  .009* 

F  =  9.848 

p  =  0.076 

F  =  3.763 

p  =  0.196 

F=  1.872 

Turbulence 

p  =  0.168 

F  =  2.158 

p  <  .01* 

F=  162.899 

p  <  .01* 

F  =  37.977 

p  =  0.007 

F=  10.302 

Haptics  * 

Turbulence 

(Interaction) 

p  =  0.601 

F  =  0.288 

p  =  0.782 

F  =  0.80 

p  =  0.430 

F  =  0.666 

p  =  0.450 

F  =  .611 
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DISCUSSION 


Effects  of  Haptics  versus  Difficulty 

From  Table  HI,  a  number  of  effects  can  be  gleaned  from  the  data: 

The  percent  of  crashes  was  significantly  reduced  by  the  haptics  condition  being  on.  To 
view  the  results  of  Table  III  more  clearly,  Figure  9  illustrates  these  key  variables  with  the 
normalized  means  of  the  percent  crashes,  Xrms.  Yrms  and  Zveiocity  displayed.  From  both 
Table  IE  and  Figure  9,  it  is  obvious  that  the  haptics  condition  being  on  significantly 
reduced  the  percent  of  crashes  (about  17  percent)  when  averaged  across  all  other 
conditions.  The  Xrms  variable  was  related  to  performance  at  landing  and  both  task 
difficulty  and  haptics  had  the  proper  influence;  e.g.  greater  task  difficulty  produced 
higher  Xrms  at  landing.  The  haptics  condition  being  on  reduced  the  Xrms  condition  since 
this  error  vector  and  the  haptic  force  reflection  algorithm  both  occurred  in  the  lateral 
direction.  For  the  Yrms  condition,  the  haptics  condition  active  did  not  have  a  significant 
reduction.  This  may  be  expected  since  the  haptic’s  effect  is  orthogonal  to  the  Y  direction. 
For  the  Zveiocity  Variable,  the  haptics  condition  being  on  did  not  have  a  significant  effect 
and  difficulty  seemed  to  have  a  lesser  influence.  The  interaction  terms  (bottom  row  of 
Table  El)  seem  well  behaved,  except,  possibly,  for  the  variable  X-RMS.  This  variable 
was  close  to  having  an  interaction  (p=.072)  since  both  the  difficulty  measure  and  haptics 
occurred  in  the  lateral  direction. 

Since  the  main  effects  are  shown  in  Table  El,  subsets  of  these  data  will  now  consider 
specific  tests  to  examine  other  possible  relationships  in  the  data. 
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Effects  of  Difficulty  versus  Turbulence 


With  reference  to  Table  IV,  if  the  turbulence  condition  was  high,  both  XRMS  and 
YRMS  were  affected  (when  averaged  over  all  difficulty  conditions).  Also  both  |R- 
velocityl  and  |R-acceleration|  were  higher  if  the  turbulence  was  high.  This  seems 
plausible  with  an  increased  turbulence  factor,  the  orientation  variable  R  (R=roll  of  an 
aircraft)  should  be  affected.  The  interactive  terms  (bottom  row  of  Table  IV)  showed  no 
major  effects  since  difficulty  and  turbulence  were  orthogonal  variables  that  were 
independent  of  each  other. 

Effects  of  Haptics  versus  Turbulence 

With  reference  to  Table  V,  when  looking  within  the  turbulence  data,  the  percent 
crashes  were  not  affected  (when  averaged  over  difficulty).  The  variable  X-RMS, 
however,  was  influenced  by  both  haptics  being  on  and  turbulence  high.  The  Y-RMS 
variable  was  less  affected  by  the  haptics  variable  but  more  so  by  turbulence.  The  haptic 
condition  worked  in  a  direction  that  was  orthogonal  to  the  Y  error.  The  |R-velocity|  term 
was  affected  by  turbulence  only.  This  is  consistent  with  Table  IV. 
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CONCLUSIONS 


From  a  performance  perspective,  certain  key  landing  variables  were  significantly 
influenced  by  the  “on”  condition  of  the  haptic  stick,  which  was  generally  in  a  direction  to 
reduce  the  lateral  touchdown  landing  error  from  the  optimum  point.  This  worked  best  and 
consistently  in  the  same  direction  that  the  haptic  force  was  applied  (X  direction). 

A  recommendation  for  future  studies  would  be  to  activate  the  haptic  stick  in  both  the 
lateral  and  longitudinal  axis.  Also  by  varying  the  degree  of  visual  immersion  as  an 
independent  variable  of  interest,  the  benefits  provided  by  the  visual  field  would  be  better 
understood  in  relationship  to  the  haptic  manipulandum. 
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APPENDIX  A  -  Subjective  Assessment  Questionnaire 


CONSENT  TO  PARTICIPATE  IN  RESEARCH 
VERITAS  FACILITY 
DEPARTMENT  OF  PSYCHOLOGY 
WRIGHT  STATE  UNIVERSITY 
DAYTON,  OHIO  45435 

The  purpose  of  this  research  study  is  to  obtain  data  on  auditory,  visual  and/or  haptic  performance  in  virtual 
environments.  These  data  will  be  used  to  guide  the  design  of  virtual  environment  systems  that  will  produce  more 
realistic  experiences  for  users. 

During  this  study,  I  will  sit  quietly,  while  visual  images  and/or  sounds,  either  stationary  or  moving,  are  presented.  In 
response  to  these  stimuli,  I  will  make  verbal  statements  or  make  gestural  motions  with  my  hands.  The  experimental 

session  will  take  approximately _ hours.  I  understand  that  I  will  be  compensated  at  a  rate  of _ for  my 

participation. 

In  some  of  the  experiments,  I  may  be  seated  inside  a  virtual  environment  facility  called  “the  CAVE”  composed  of  four 
10x10-  foot  display  screens  in  the  shape  of  a  square  room.  In  some  experiments  I  may  see  only  visual  images  (on  the 
large  display  screens  and/or  the  floor ,  or  computer  monitors  or  both)  without  sound,  I  may  hear  sounds  (either  over 
headphones  or  over  loud  speakers)  or  I  may  see  images  and  hear  sounds  together.  I  may  wear  thin  cloth  gloves  to 
measure  my  hand  gestures  (e.g.  pinching  motions).  I  may  wear  liquid  crystal  (LCD)  shutter  glasses,  like  large 
sunglasses  to  see  3D  images,  but  can  wear  these  shutter  glasses  over  any  eyeglasses  or  contact  lenses  I  may  normally 
wear.  The  shutter  glasses  or  headphones  may  have  a  commercial  magnetic  tracking  sensor  attached  to  it  that  is  used  to 
measure  and  record  the  position  of  my  head.  The  cloth  gloves  may  have  the  same  type  of  sensors  attached  to  measure 
and  record  the  positions  of  my  hands.  In  some  experiments  I  will  use  a  hand  controller  or  joystick  to  interact  with  and 
control  the  motion  or  behavior  of  objects  in  the  virtual  environment. 

I  understand  that  any  information  about  me  obtained  from  this  study  will  be  kept  strictly  confidential  and  that  I  will  not 
be  identified  by  name  in  any  report  or  publication. 

I  realize  that  I  may  experience  symptoms  like  motion  sickness,  similar  to  what  I  might  experience  riding  in  a  car  or 
after  a  carnival  ride.  If  I  experience  symptoms,  I  should  immediately  inform  the  experimenter  through  the  intercom.  In 
addition,  I  will  be  tested  for  motion  sickness  symptoms  before  and  after  the  experiment.  I  understand  that  if  the 
symptoms  have  not  gone  away  after  1  hour,  I  will  be  advised  not  to  drive  a  car  for  24  hours  and  a  ride  will  be  provided. 
The  sounds  I  hear  would  never  be  harmful  or  uncomfortably  loud.  If  the  sounds  are  uncomfortable,  I  should 
immediately  remove  the  headphones  and  notify  the  experimenter  through  the  intercom.  If  viewing  the  images  causes 
any  eye  strain,  I  should  immediately  notify  the  experimenter  through  the  intercom. 

I  realize  that  participation  in  this  research  is  completely  voluntary.  I  understand  that  I  am  free  to  refuse  to  participate  in 
this  study  or  to  withdraw  at  any  time.  There  is  no  penalty  of  any  kind  for  either  non-participation  or  withdrawal. 

A  summary  of  the  results  of  the  experiment  may  be  requested  by  contacting  the  researchers  listed  below.  The  results  of 
this  study  will  be  available  on  approximately _ . 

If  I  have  concerns  or  questions  about  the  research  I  understand  I  can  contact  the  Principal  Investigator,  Robert  H. 

Gilkey  at  775-337  John  Hach,  Co-Investigator  775-2574  or  through  the  Wright  State  University,  Department  of 
Psychology  (775-2391). 

My  signature  below  indicates  that  I  consent  to  participate  in  this  research  investigation. 

_ Date _ 


Name  (Please  Print  Neatly). 
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